Assistance tools for the evaluation of machine learning algorithm performance: The decision tree-based tools developed by the Medical Imaging and Data Resource Center (MIDRC) Technology Development Project (TDP) 3c effort

MIDRC was created to facilitate machine learning research for tasks including early detection, diagnosis, prognosis, and assessment of treatment response related to the COVID-19 pandemic and beyond. The purpose of the Technology Development Project (TDP) 3c is to create resources to assist researchers in evaluating the performance of their machine learning algorithms. An interactive decision tree has been developed, organized by the type of task that the machine learning algorithm is being trained to perform. The user can select information such as: (a) the type of task, (b) the nature of the reference standard, and (c) the type of the algorithm output. Based on the user responses, they can obtain recommendations regarding appropriate performance evaluation approaches and metrics, including literature references, short video tutorials, and links to available software. Five tasks have been identified for the decision tree: (a) classification, (b) detection/localization, (c) segmentation, (d) time-to-event analysis, and (e) estimation. As an example, the classification branch of the decision tree includes binary and multi-class classification tasks and provides suggestions for methods and metrics as well as software recommendations, and literature references for situations where the algorithm produces either binary or non-binary (e.g., continuous) output and for reference standards with negligible or non-negligible variability and unreliability. The decision tree has been made publicly available on the MIDRC website to assist researchers in conducting task-specific performance evaluations, including classification, detection/localization, segmentation, estimation, and time-to-event tasks. © COPYRIGHT SPIE. Downloading of the is permitted for personal use only.

Sequestration methodology in practice through evaluation of joint demographic distributions of 54,185 patients in the Medical Imaging and Data Resource Center (MIDRC) data commons

The Medical Imaging and Data Resource Center (MIDRC) is a multi-institutional effort to accelerate medical imaging machine intelligence research and create a publicly available data commons as well as a sequestered commons for performance evaluation of algorithms. This work sought to evaluate the currently implemented methodology for apportioning data to the public and sequestered data commons by investigating the resulting distributions of joint demographic characteristics between the public and sequestered commons. 54,185 patients whose de-identified imaging studies and metadata had been submitted to MIDRC were previously separated into public and sequestered commons using a multi-dimensional stratified sampling method, resulting in 41,556 patients (77%) in the public commons and 12,629 patients (23%) in the sequestered commons. To compare the balance obtained in the joint distributions of patient characteristics from use of the developed sequestration method, patients from each commons were separated into bins, representing a unique combination of the demographic variables of COVID-19 status, age, race, and sex assigned at birth. The joint distributions of patients were visualized, and the absolute and percent difference in each bin from an exact 77:23 split of the data were calculated. Results indicated 75.9% of bins obtained differences of less than 15 patients, with a median difference of 3.6 from the total data for both public and sequestered commons. Joint distributions of patient characteristics in both the public and sequestered commons closely matched each other as well as that of the total data, indicating the sequestration by stratified sampling method has operated as intended. © 2023 SPIE.

Increased shunt and dead space in recovered COVID-19 pneumonitis patients

Introduction/Aim: We previously reported impaired pulmonary gas exchange in acute COVID-19 patients resulting from both increased intrapulmonary shunt (SH) and increased alveolar dead space (AD) 1 . The present study quantifies gas exchange in recovered patients. Method(s): Unvaccinated patients diagnosed with acute COVID-19 infection (March-December 2020) were studied 15 to 403 days post first SARS-CoV-2 positive PCR test. Demographic, anthropometric, acute disease severity and comorbidity data were collected. Breathing room air, steady-state exhaled gas concentrations were measured simultaneously with arterial blood gases. Alveolar CO 2 and O 2 (P A CO 2 and P A O 2 ;mid-exhaled volume) determined;AaPO2, aAPCO2, SH% and AD% calculated. 2 Results: We studied 59 patients (33 males, Age: 52[38-61] years, BMI: 28.8[25.3-33.6] kg/m 2 ;median[IQR]). Co-morbibities included asthma (n = 2), cardiovascular disease (n = 3), hypertension (n = 12), and diabetes (n = 9);14 subjects smoked;44 had experienced mild-moderate COVID-19 (NIH category 1-2), 15 severe-critical disease (NIH category 3-5). PaCO 2 was 39.4[35.6-41.1] mmHg, PaO 2 92.1[87.1-98.2] mmHg;P A CO 2 32.8[28.6-35.3] mmHg, P A O 2 112.9[109.4-117.0] mmHg, AaPO 2 18.8[12.6-26.8] mmHg, aAPCO 2 5.9[4.3-8.0] mmHg, SH 4.3 [2.1-5.9]% and AD 16.6 [12.6-24.4]%. 14% of patients had normal SH (<5%) and AD (<10%);1% abnormal SH and normal AD;36% both abnormal SH and AD;49% normal shunt and abnormal AD. Previous severe-critical disease was a strong independent predictor for increased SH (OR 14.8[2.28-96], [95% CI], p < 0.01), increasing age weakly predicted increased AD (OR 1.18[1.01, 1.37], p < 0.04). Time since infection, BMI and comorbidities were not significant predictors (all p > 0.11). Conclusion(s): Prior COVID-19 was associated with increased intrapulmonary shunt and/or increased alveolar dead space in 86% of this cohort up to ~13 months post infection, with those with more severe acute disease, and older patients, at greater risk. Increased intrapulmonary shunt suggests persistent alveolar damage, while increased alveolar dead space may indicate persistent pulmonary vascular occlusion.

Impact of the COVID-19 pandemic on blood services operations: Korean experience

The coronavirus disease 19 (COVID-19) pandemic had a profound impact on blood services operations in Korea. Blood collection was affected due to decrease in donor availability caused by avoidance of public places, social distancing policies, and cancellation of blood drives. The negative impact on blood collection was more pronounced with the COVID-19 pandemic than with other outbreaks experienced previously such as the influenza (H1N1) outbreak or the Middle East respiratory virus (MERS) pandemic. To cope with the blood shortage, campaigns to appeal for blood donation, raise public awareness on the importance of blood donation and gain donor's confidence in safe blood donation were implemented using mass communication media such as TV and radio broadcasting as well as postings on various social media platforms. Upon Korean Red Cross Blood Services's (KRCBS) request, the Ministry of Health and Welfare (MoHW) approved the relaxation of the geographical restrictions regarding indigenous malaria thus enabling collection of more than 23,000 units of whole blood. To mitigate even a theoretical risk of transfusion-transmission of SARS-CoV-2 via blood donation from pre-symptomatic COVID-19 donors, the KRCBS received the data on COVID-19 identified cases from the Korean Disease Control and Prevention Agency (KDCA) from the early get-go of the pandemic for cross referencing to donors for further recipient investigation and recall of blood products not transfused. Communication with donors, staff members, national health authorities, hospital customers and other stakeholders was and remains of utmost importance to respond to this unprecedented situation which is still ongoing.Copyright © Annals of Blood. All rights reserved.

The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19

Data visualization has become an increasingly important means of effective data communication and has played a vital role in broadcasting the progression of COVID-19. Accessible data representations, however, have lagged behind, leaving areas of information out of reach for many blind and visually impaired (BVI) users. In this work, we sought to understand (1) the accessibility of current implementations of visualizations on the web;(2) BVI users' preferences and current experiences when accessing data-driven media;(3) how accessible data representations on the web address these users' access needs and help them navigate, interpret, and gain insights from the data;and (4) the practical challenges that limit BVI users' access and use of data representations. To answer these questions, we conducted a mixed-methods study consisting of an accessibility audit of 87 data visualizations on the web to identify accessibility issues, an online survey of 127 screen reader users to understand lived experiences and preferences, and a remote contextual inquiry with 12 of the survey respondents to observe how they navigate, interpret, and gain insights from accessible data representations. Our observations during this critical period of time provide an understanding of the widespread accessibility issues encountered across online data visualizations, the impact that data accessibility inequities have on the BVI community, the ways screen reader users sought access to data-driven information and made use of online visualizations to form insights, and the pressing need to make larger strides towards improving data literacy, building confidence, and enriching methods of access. Based on our findings, we provide recommendations for researchers and practitioners to broaden data accessibility on the web. © 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Navigating a crisis: do exploration and exploitation help SMEs when they respond to COVID-19 disruption?

This study investigates the validity of exploitation and exploration when small- and medium-sized enterprises (SMEs) navigate a highly uncertain time. Although balancing the two strategies has been thought to lead to improved firm performance in general, a combined approach appears to be problematic for SMEs due to a lack of feasibility. We theorise that the effectiveness may vary depending on a fit between the strategies and the environmental contingencies. In doing so, we considered two potential environmental contingencies of a crisis for SMEs: loss of demand and loss of supply. To put our theory to the test, we gathered 224 responses from business leaders and key individuals from Korean start-ups and tested the effectiveness of crisis management strategies. Our findings support the validity of both exploitation and exploration when firms face a loss of demand, but not a loss of supply. It implies that the effectiveness of exploration and exploitation is contingent upon a specific form of crisis experienced at the firm level. © KOSIME, ASIALICS, STEPI 2023.

Community engagement and data quality: best practices and lessons learned from a citizen science project on birdsong.

Citizen Science (CS) is a research approach that has become popular in recent years and offers innovative potential for dialect research in ornithology. As the scepticism about CS data is still widespread, we analysed the development of a 3-year CS project based on the song of the Common Nightingale (Luscinia megarhynchos) to share best practices and lessons learned. We focused on the data scope, individual engagement, spatial distribution and species misidentifications from recordings generated before (2018, 2019) and during the COVID-19 outbreak (2020) with a smartphone using the 'Naturblick' app. The number of nightingale song recordings and individual engagement increased steadily and peaked in the season during the pandemic. 13,991 nightingale song recordings were generated by anonymous (64%) and non-anonymous participants (36%). As the project developed, the spatial distribution of recordings expanded (from Berlin based to nationwide). The rates of species misidentifications were low, decreased in the course of the project (10-1%) and were mainly affected by vocal similarities with other bird species. This study further showed that community engagement and data quality were not directly affected by dissemination activities, but that the former was influenced by external factors and the latter benefited from the app. We conclude that CS projects using smartphone apps with an integrated pattern recognition algorithm are well suited to support bioacoustic research in ornithology. Based on our findings, we recommend setting up CS projects over the long term to build an engaged community which generates high data quality for robust scientific conclusions. Supplementary Information: The online version contains supplementary material available at 10.1007/s10336-022-02018-8.

MM Algorithms to Estimate Parameters in Continuous-time Markov Chains (preprint)

Continuous-time Markov chains (CTMCs) are popular modeling formalism that constitutes the underlying semantics for real-time probabilistic systems such as queuing networks, stochastic process algebras, and calculi for systems biology. Prism and Storm are popular model checking tools that provide a number of powerful analysis techniques for CTMCs. These tools accept models expressed as the parallel composition of a number of modules interacting with each other. The outcome of the analysis is strongly dependent on the parameter values used in the model which govern the timing and probability of events of the resulting CTMC. However, for some applications, parameter values have to be empirically estimated from partially-observable executions. In this work, we address the problem of estimating parameter values of CTMCs expressed as Prism models from a number of partially-observable executions. We introduce the class parametric CTMCs -- CTMCs where transition rates are polynomial functions over a set of parameters -- as an abstraction of CTMCs covering a large class of Prism models. Then, building on a theory of algorithms known by the initials MM, for minorization-maximization, we present iterative maximum likelihood estimation algorithms for parametric CTMCs covering two learning scenarios: when both state-labels and dwell times are observable, or just state-labels are. We conclude by illustrating the use of our technique in a simple but non-trivial case study: the analysis of the spread of COVID-19 in presence of lockdown countermeasures.

GEOGRAPHIC PROXIMITY TO NEIGHBORHOOD RESOURCES AND DEPRESSIVE SYMPTOMS AMONG KOREAN OLDER ADULTS

Given the importance of geographic proximity to neighborhood resources especially during the COVID-19 pandemic, this study examine whether the relationship between geographic proximity to neighborhood resources (e.g. hospitals, public transportation, etc.) and depressive symptoms varied by geographic location (i.e., rural vs. urban areas) among older adults in South Korea and whether this relationship was mediated by participation in social activities (e.g. education, club, community, etc.). The nationally representative samples, Korean older adults aged 65 or older, were drawn from the 2020 Survey of Living Conditions and Welfare Needs of Korean Older Persons (N=9,732, Urban=6,975, Rural=2,757). Hierarchical regression models, Baron and Kenny's steps, and Sobel Test for the mediation effect were conducted. Results showed that geographic proximity was negatively associated with depressive symptoms in urban areas (B=-.041, p<.001), while positively associated in rural areas (B=.034, p<.01). Participation in social activities partially mediated the relationship in urban areas (Z=-2.162, p<.05), while there was no significant mediation effect in rural areas. Additionally, geographic proximity to hospitals or public transportation was significantly associated with depressive symptoms in rural areas. The findings suggest that geographic proximity to neighborhood resources helps older adults reduce social isolation, which may improve mental health of older adults living in urban areas during the pandemic. However, geographic proximity to neighborhood resources could make older adults living in rural areas become depressed, emphasizing that the characteristics of the urban and rural areas need to be considered to create an aged-friendly environment.

Real-Time Facial Animation Generation on Face Mask

In light of the COVID-19 pandemic, wearing a mask is crucial to avoid contracting infectious diseases. However, wearing a mask is known to impair communication functions. This study aims to address the communication difficulties caused by wearing a mask and provide a strategy for aiding in understanding the speaker's speech through facial animation. Facial animation is generated in real-time, and upper facial information is processed to detect the speaker's emotions, generating a lower facial expression. In addition, the system detects the mask's shape and enables accurate registration in the proper position. This technology can improve communication and alleviate challenges associated with communication between persons wearing face masks. © 2022 Owner/Author.

An investigation of safe and near-optimal strategies for prevention of Covid-19 exposure using stochastic hybrid models and machine learning

In this work investigate the use of stochastic hybrid models, statistical model checking and machine learning to analyze, predict and control the rapid spreading of Covid-19. During the pandemic numerous studies using stochastic models have been produced. Most of these studies are used to predict the effect of some restrictions. In contrast, in this paper we focus on the synthesis of strategies which prevent Covid-19 spreading. The computed strategies provide valuable information which can be used by the authorities to design new and more specific restrictions. We consider two large case studies that develop in the Copenhagen area in Denmark. Our experiments show that the computed strategies significantly prevent Covid-19 spreading, and thus provide valuable information e.g. expected social distance to minimize Covid-19 spreading. On the technical side, we demonstrate the applicability of analytical methods for preventing the spreading of Covid-19 in large scenarios.

Enhancing Code Similarity with Augmented Data Filtering and Ensemble Strategies

Although COVID-19 has severely affected the global economy, information technology (IT) employees managed to perform most of their work from home. Telecommuting and remote work have promoted a demand for IT services in various market sectors, including retail, entertainment, education, and healthcare. Consequently, computer and information experts are also in demand. However, producing IT, experts is difficult during a pandemic owing to limitations, such as the reduced enrollment of international students. Therefore, researching increasing software productivity is essential;this study proposes a code similarity determination model that utilizes augmented data filtering and ensemble strategies. This algorithm is the first automated development system for increasing software productivity that addresses the current situation—a worldwide shortage of software dramatically improves performance in various downstream natural language processing tasks (NLP). Unlike general-purpose pre-trained language models (PLMs), CodeBERT and GraphCodeBERT are PLMs that have learned both natural and programming languages. Hence, they are suitable as code similarity determination models. The data filtering process consists of three steps: (1) deduplication of data, (2) deletion of intersection, and (3) an exhaustive search. The best mating (BM) 25 and length normalization of BM25 (BM25L) algorithms were used to construct positive and negative pairs. The performance of the model was evaluated using the 5-fold cross-validation ensemble technique. Experiments demonstrate the effectiveness of the proposed method quantitatively. Moreover, we expect this method to be optimal for increasing software productivity in various NLP tasks. © 2022, Politeknik Negeri Padang. All rights reserved.

Virtual interprofessional team care planning and communication for chronic pain management: An educational model

Ideal patient care involves interprofessional collaboration and therefore emphasizes the importance of communicating how roles and responsibilities differ to create a team environment critical for providing optimal patient care. In light of the ongoing opioid epidemic associated with chronic pain, this interprofessional simulation focused on utilizing an interprofessional team approach to recognize the biopsychosocial and pharmacologic aspects of chronic pain management through creation of a patient-centered care plan using a virtual platform. Virtual IPE events can be performed by institutions with limited access to other healthcare disciplines for remote learning opportunities and can be adapted to develop comprehensive strategies to evaluate effectiveness of learning interventions among varied disciplines. Participants from the Schools of Medicine, Nursing, Pharmacy, and Health Related Professions, including occupational and physical therapy, participated in the virtual simulation event. The format was chosen to adhere to current COVID-19 safety guidelines and facilitate easier scheduling between disciplines. The event included individual pre-work through an online learning management system leading to a 2-h virtual simulation event. Interprofessional Education Collaborative, or IPEC developed competencies focused on communication and teamwork to establish activity objectives. International Association for the Study of Pain, or IASP, pain curriculum outlines provided additional objectives and guided presented information on best practice approaches for interprofessional pain management. Objectives were evaluated through peer team feedback, peer discipline feedback, and assessment of the comprehensive team care plan that consisted of pharmacologic and nonpharmacologic pain management strategies. Programmatic review demonstrated students were able to have effective communication that led to a holistic patient care plan at the end of this activity.

Telehealth care before and during COVID-19: trends and quality in a large health system.

Objective: COVID-19 accelerated telehealth use to ensure care delivery, but there is limited data on the patient perspective. This study aimed to examine telehealth visit uptake before and during COVID-19 and correlates of patient satisfaction and interest in future telehealth visits. Materials and Methods: This was a cross-sectional observational study between October 2019 and April 2020. Participants included patients who completed satisfaction surveys following telehealth visits. Results: A total of 8930 patients completed the satisfaction survey using 4-point Likert Scales. Multivariable, hierarchical, cumulative logit models were constructed to examine correlates of satisfaction with quality of care and interest in future telehealth visits. Most patients were satisfied with the patient portal, video quality, and instructions (92.7%-96.8%). Almost half reported saving 1-2 h (46.9%). Correlates positively associated with quality of care and interest in future telehealth visits were ease of patient portal (odds ratio [OR], 1.43, 95% confidence interval [CI], 1.30-1.58; OR, 1.56, 95% CI, 1.41-1.73, respectively), video quality (OR, 1.62, 95% CI, 1.50-1.75; OR, 1.26, 95% CI, 1.16-1.37, respectively), instructions (OR, 5.62, 95% CI, 5.05-6.26; OR, 1.80, 95% CI, 1.62-2.01, respectively), and time saved (>4 h: OR, 1.69, 95%,CI, 1.22-2.34; OR, 3.49, 95% CI, 2.47-4.93, respectively). Being seen after the COVID-19 surge in telehealth (OR, 0.76, 95% CI, 0.63-0.93) or by providers with higher visit volume (OR, 0.71, 95% CI, 0.60-0.85) was associated with lower interest in future telehealth visits. Conclusions: Patients expressed relatively high satisfaction levels with telehealth. Better technical quality, quality of instructions, and greater time saved were associated with higher satisfaction ratings. To maintain interest in future telehealth use and improve the patient experience, we must enhance the quality of telehealth delivery platforms and instructions provided to patients.